UNIX Resource Managers: Capacity Planning and Resource Issues

The latest generation of UNIX includes facilities for automatically managing system resources such as processors, disks, and memory. Traditional UNIX time-share schedulers attempt to be fair to all users by employing a round-robin policy for allocating CPU time. Unfortunately, a loophole exists whereby the scheduler can be biased in favor of a greedy user running many short CPU-demand processes. This loophole is not a defect but an intrinsic property of the round-robin scheduler that ensures responsiveness to the brief CPU demands associated with multiple interactive users. The new generation of UNIX constrains the scheduler to be equitable to all users regardless of the number of processes each may be running. This "fair-share" scheduling draws on the concept of pro rating resource "shares" across users and groups and then dynamically adjusting CPU usage to meet those share proportions. This simple administrative notion of statically allocating shares, however, belies the potential consequences for performance as measured by user response time and service level targets. We examine several simple share allocation scenarios and the surprising performance consequences with a view to establishing some guidelines for allocating shares. INTRODUCTION Capacity planning is becoming an increasingly important consideration for UNIX system administrators as multiple workloads are consolidated onto single large servers, and the procurement of those servers takes on the exponential growth of internet-based business. The latest implementations of commercial UNIX to offer mainframe-style capacity management on enterprise servers include: AIX Workload Manager (WLM), HP-UX Process Resource Manager (PRM), Solaris Resource Manager (SRM), SGI and Compaq. The ability to manage server capacity is achieved by making significant modifications to the standard UNIX operating system so that processes are inherently tied to specific users. Those users, in turn, are granted only a certain fraction of system resources. Resource usage is monitored and compared with each userÕs grant to ensure that the assigned entitlement constraints are met. Shared system resources that can be managed in this way include: processors, memory, and mass storage. Prima face, this appears to be exactly what is needed for a UNIX system administrator to manage server capacity. State of the art UNIX resource management, however, is only equivalent to that which was provided on mainframes more than a decade ago but it had to start somewhere. The respective UNIX operations © 2000 Performance Dynamics Consultinga. All Rights Reserved. No part of this document may be reproduced in any way without the prior written permission of the author. Permission is granted to SAGE-AU to publish this article for the Conference in July 2000. SAGE-AU Conference, Gold Coast, Australia, July 3-7 2000 2 manuals spell out the virtues of each vendor's implementation, so the purpose of this paper is to apprise the sysadm of their limitations for meeting certain performance requirements. For example, an important resource management option is the ability to set entitlement caps when actual resource consumption is tied to other criteria such as performance targets and financial budgets. Another requirement is the ability to manage capacity across multiple systems e.g., shared-disk and NUMA clusters. As UNIX system administrators become more familiar with the new automated resource management facilities, it also becomes important that they learn some of the relevant capacity planning methodologies that have already been established for mainframes. In that spirit, we begin by clearing up some of the confusion that has surrounded the motivation and the terminology behind the new technology. The common theme across each of the commercial implementations is the introduction of a fair-share scheduler. After reviewing the alternative scheduler, we examine some of the less obvious potential performance pitfalls. Finally, we present some capacity planning guidelines for migrating to automated UNIX resource management. LOOPHOLES IN TIME UNIX is fundamentally a time-share (TS) operating system [Ritchie & Thompson 1978] aimed at providing responsive service to multiple interactive users. Put simply, the goal of generic UNIX is to create the illusion for each user that they are the only one accessing system resources. This illusion is controlled by the UNIX time-share scheduler. A time-share scheduler attempts to give all active processes equal access to processing resources by employing a round-robin scheduling policy for allocating processor time. Put simply, each process is placed in a run-queue and when it is serviced, it gets a fixed amount of time running on the processor. This fixed time is a called a service quantum (commonly set to 10 milliseconds). Processes that demand less than the service quantum, generally complete processor service without being interrupted. However, a process that exceeds its service quantum has its processing interrupted and is returned toward the back of the run-queue to await further processing. A natural consequence of this scheduling policy is that processes with shorter processing demands are favored over longer ones because the longer demands are always preempted when their service quantum expires. Herein lies a loophole. The UNIX scheduler is biased intrinsically in favor of a greedy user who runs many short-demand processes. This loophole is not a defect, rather it is a property of the round-robin scheduler that implements time-sharing which, in turn, guarantees responsiveness to the short processing demands associated with multiple interactive users. Figure 1 shows this effect explicitly. In Fig. 1, two users are distinguished as owning heavy and light processes in terms of their respective processing demands. Below 10 processes, the heavy user sees a better transaction throughput than the light user. Beyond 10 processes the throughput of the light user begins to dominate the heavy user; in spite of the fact that both users are adding more processes into the system! SAGE-AU Conference, Gold Coast, Australia, July 3-7 2000 3 Fig. 1. Generic UNIX intrinsically favors shorter service-demand processes. Note that the UNIX scheduler schedules processes, not users. Under a time-share scheduler there is no way for users to assert that they would like a more equitable allocation of processing resources to defeat greedy consumption by other users. The new generation of UNIX addresses this loophole by implementing a fair-share (FS) scheduler [Kay & Lauder 1988] that equitable processing to all users regardless of the number of processes each may be running. WHAT IS FAIR? To define "fairness", the FS-based scheduler has a control parameter that allows the sysadm to pro rate literal "shares" across users and groups of users. The FS scheduler then dynamically adjusts processor usage to match those share proportions. Similarly, shares can also be allocated to control memory and disk I/O resource consumption. For the remainder of this paper, we'll confine the discussion to processor consumption because it is easiest to understand and has the most dynamic consequences. In the context of computer resources, the word fair is meant to imply equity, not equality in resource consumption. In other words, the FS scheduler is equitable but not egalitarian. This distinction becomes clearer if we consider shares in a publicly owned corporation. Executives usually hold more equity in a company than other employees. Accordingly, the executive shareholders are entitled to a greater percentage of company profits. And so it is under the FS scheduler. The more equity you hold in terms of the number of allocated resource shares, the greater the percentage of server resources you are entitled to at runtime. The actual number of FS shares you own is statically allocated by the UNIX sysadm. This share allocation scheme seems straightforward. However, under the FS scheduler there are some significant differences from the way corporate shares work. Although the shares allocated to you is a fixed number, the proportion of resources you Time-Share Scheduling 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0 5 1 0 1 5 2 0 2 5 3 0